Avoid selecting prerelease Python installations without opt-in #7300

zanieb · 2024-09-11T18:17:55Z

Similar to our semantics for packages with pre-release versions.

We will not use prerelease versions unless there are only prerelease versions available, a specific version is requested,
or the prerelease version is found in a reasonable source (active environment, explicit path, etc. but not PATH).

For example, uv python install 3.13 && uv run python --version will no longer use 3.13.0rc2 unless that is the only Python version available, --python 3.13 is used, or that's the Python version that is present in .venv.

zanieb · 2024-09-11T18:20:26Z

This change could be breaking for users with a 3.13 prerelease installed, but the existing behavior does not match our usual version semantics.

zanieb · 2024-09-11T18:45:56Z

crates/uv-python/src/discovery.rs

+    /// Whether a pre-release Python installation from the source should be used without opt-in.
+    pub(crate) fn allows_prereleases(self) -> bool {
+        match self {
+            Self::Managed | Self::Registry | Self::SearchPath | Self::MicrosoftStore => false,
+            Self::CondaPrefix
+            | Self::ProvidedPath
+            | Self::ParentInterpreter
+            | Self::ActiveEnvironment
+            | Self::DiscoveredEnvironment => true,
+        }
+    }


I didn't have this at first and the Python 3.13 system test failed at https://github.com/astral-sh/uv/actions/runs/10817366263/job/30011226637

It's still failing :/ presumably because there's another Python version available

Looking at a fix in #7306

See #7300 (comment) too

Previously we only had coverage on Windows e.g. for more test coverage in cases like #7300 (comment) I picked Linux over macOS because the 3.13 prereleases are not available via HomeBrew?

…ystem_python` (#7306) Needed for #7300

Follows #7278 Closes #7280 `uv python list` should show installed pre-release versions, even though we don't select them by default (as defined by #7300 and #7278)

Similar to our sementics for packages with pre-release versions, we will prefer non-prerelease versions unless there are only prerelease versions available.

zanieb · 2024-09-11T19:55:53Z

.github/workflows/ci.yml

-        run: python3.13 scripts/check_system_python.py --uv ./uv
+        run: python3.13 scripts/check_system_python.py --uv ./uv --python 3.13


A similar change was required on Windows.

uv will now prefer another installation on the system, if it exists. In this case, Python 3.10 is available and breaks our expectations.

henryiii · 2024-09-16T19:16:28Z

This also seems to break if you've set VIRTUAL_ENV to a venv with 3.13 in it? That's what cibuildwheel does; build[uv] is now broken in cibuildwheel (and we are pushing to get everyone to build 3.13 wheels now before release).

I think this also might break nox, tox, etc.

zanieb · 2024-09-16T22:06:55Z

For prosperity, that isn't the case per the test in #7437

henryiii · 2024-09-17T14:36:09Z

Yes, the issue wasn't VIRTUAL_ENV, but rather that we were making the venv assuming it would use the Python on the top of the PATH. cibuildwheel 2.21.1 fixes this.

It's really weird, though, that it no longer takes the Python at the top of the PATH when --system is used. If someone goes to the trouble to put Python in the PATH, intentionally skipping over it is really surprising; and if you type python, you don't get the one you just installed to. For example:

- uses: actions/setup-python@v5
  with:
    python-version: ${{ matrix.python-version }}
    allow-prereleases: true
- uses: astral-sh/setup-uv@v3
- run: uv pip install --system some_lib
- run: python -c "import some_lib"

now fails only on 3.13 because uv installed to the 3.12 instead of the 3.13 that's been prepared by setup-python. The python above is 3.13, while uv is skipping it and installing into 3.12. (I've seen this failure somewhere but have forgotten which repos it affected).

I understand this makes sense with managed Python (where you have a bunch of Pythons and uv's just picking out the best one), but it is very surprising to have it do this to the Python on PATH.

As we support more complex Python discovery behaviors such as: - #7431 - #7335 - #7300 `Any` is no longer accurate, we actually are looking for a reasonable default Python version to use which may exclude the first one we find. Separately, we need the idea of `Any` to improve behavior when listing versions (e.g., #7286) where we do actually want to match _any_ Python version. As a first step, we'll rename `Any` to `Default`. Then, we'll introduce a new `Any` that actually behaves as we'd expect.

This MR contains the following updates: | Package | Update | Change | |---|---|---| | [astral-sh/uv](https://github.com/astral-sh/uv) | patch | `0.4.9` -> `0.4.13` | MR created with the help of [el-capitano/tools/renovate-bot](https://gitlab.com/el-capitano/tools/renovate-bot). **Proposed changes to behavior should be submitted there as MRs.** --- ### Release Notes <details> <summary>astral-sh/uv (astral-sh/uv)</summary> ### [`v0.4.13`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0413) [Compare Source](astral-sh/uv@0.4.12...0.4.13) ##### Enhancements - Add `socks` support ([#7503](astral-sh/uv#7503)) - Avoid warning about bad Python interpreter links for empty project environment directories ([#7527](astral-sh/uv#7527)) - Improve invalid environment warning messages ([#7544](astral-sh/uv#7544)) - Use more verbose spelling of "virtualenv" during creation ([#7523](astral-sh/uv#7523)) - Do not use a user-facing warning for "Waiting to acquire lock..." message ([#7502](astral-sh/uv#7502)) ##### Performance - Use a single buffer for hints on resolver errors ([#7497](astral-sh/uv#7497)) ##### Bug fixes - Allow Python pre-releases to be used if they are first on the `PATH` ([#7470](astral-sh/uv#7470)) - Avoid deleting the project environment directory if it is not a virtual environment ([#7522](astral-sh/uv#7522)) - Do not error if the `CACHEDIR.TAG` file exists but cannot be written to ([#7550](astral-sh/uv#7550)) - Treat invalid platform as more compatible than invalid Python ([#7556](astral-sh/uv#7556)) - Use portable paths when serializing sources ([#7504](astral-sh/uv#7504)) - Compute resolver hints using the final reduced derivation tree ([#7546](astral-sh/uv#7546)) - Bump the wheel and sdist cache versions ([#7560](astral-sh/uv#7560)) - Heal cache entries with missing source distributions ([#7559](astral-sh/uv#7559)) ##### Rust libraries - Bump minimum supported Rust version from 1.80 -> 1.81 ##### Documentation - Add `UV_LINK_MODE` to Docker caching example ([#7510](astral-sh/uv#7510)) - Clarify behavior of of overrides in CLI reference ([#7537](astral-sh/uv#7537)) ### [`v0.4.12`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0412) [Compare Source](astral-sh/uv@0.4.11...0.4.12) ##### Enhancements - Allow users to provide pre-defined metadata for resolution ([#7442](astral-sh/uv#7442)) - Invalidate existing tool environments on Python interpreter mismatch ([#7451](astral-sh/uv#7451)) ##### Bug fixes - Avoid fatal error when searching for egg-info with missing directory ([#7498](astral-sh/uv#7498)) ##### Documentation - Add note on cache growth for self-hosted GitHub runners ([#5757](astral-sh/uv#5757)) ### [`v0.4.11`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0411) [Compare Source](astral-sh/uv@0.4.10...0.4.11) ##### Enhancements - Add `--no-editable` support to `uv sync` and `uv export` ([#7371](astral-sh/uv#7371)) - Add support for `--only-dev` to `uv sync` and `uv export` ([#7367](astral-sh/uv#7367)) - Add support for remaining pip-supported file extensions ([#7387](astral-sh/uv#7387)) - Generate shell completion for `uvx` ([#7388](astral-sh/uv#7388)) - Include `uv export` command in `requirements.txt` output ([#7374](astral-sh/uv#7374)) - Prune unzipped source distributions in `uv cache prune --ci` ([#7446](astral-sh/uv#7446)) - Warn when trying to `uv sync` a package without build configuration ([#7420](astral-sh/uv#7420)) - Support requests for pre-releases in the `--python` option ([#7335](astral-sh/uv#7335)) ##### Bug fixes - Avoid erroneous version warning for `.dist-info` directories ([#7444](astral-sh/uv#7444)) - Avoid removing seed packages for `uv venv --seed` environments ([#7410](astral-sh/uv#7410)) - Avoid unnecessary progress bar initializations ([#7412](astral-sh/uv#7412)) - Error when `tool.uv.sources` contains duplicate package names ([#7383](astral-sh/uv#7383)) - Include `--branch` et al when resolving unnamed URLs in `uv add` ([#7447](astral-sh/uv#7447)) - Include `dev-dependencies` in `--no-sources` invocations ([#7408](astral-sh/uv#7408)) - Include the parent interpreter in Python discovery when `--system` is used ([#7440](astral-sh/uv#7440)) - Respect `--no-sources` in PEP 723 scripts ([#7409](astral-sh/uv#7409)) - Respect `pyproject.toml` credentials from user-provided requirements ([#7474](astral-sh/uv#7474)) - Use consistent PyPI cache bucket ([#7443](astral-sh/uv#7443)) - Use unambiguous relative paths in `uv export` ([#7378](astral-sh/uv#7378)) ##### Documentation - Add documentation on platform-specific dependencies ([#7411](astral-sh/uv#7411)) - Add documentation for passing installer options on Linux ([#6839](astral-sh/uv#6839)) - Separate project data from configuration settings ([#7053](astral-sh/uv#7053)) ##### Error messages - Hint at missing `project.name` ([#6803](astral-sh/uv#6803)) - Surface dedicated `project.name` error for workspaces ([#7399](astral-sh/uv#7399)) - Remove duplicate warning for settings discovery errors ([#7384](astral-sh/uv#7384)) ### [`v0.4.10`](https://github.com/astral-sh/uv/blob/HEAD/CHANGELOG.md#0410) [Compare Source](astral-sh/uv@0.4.9...0.4.10) ##### Enhancements - Allow `uv tool upgrade --all` to continue on individual upgrade failure ([#7333](astral-sh/uv#7333)) - Support globs as cache keys in `tool.uv.cache-keys` ([#7268](astral-sh/uv#7268)) - Add Python package (`__main__.py`) support to `uv run` ([#7281](astral-sh/uv#7281)) - Add zip application support to `uv run` ([#7289](astral-sh/uv#7289)) - Add `--token` option to `self update` command ([#7279](astral-sh/uv#7279)) ##### Performance - Use `globwalk` for `cache-keys` matching ([#7337](astral-sh/uv#7337)) ##### Bug fixes - Always treat archive-like requirements as local files ([#7364](astral-sh/uv#7364)) - Apply `--no-install` options when constructing resolution ([#7277](astral-sh/uv#7277)) - Avoid clobbering existing `py.typed` files contents in `uv init` ([#7338](astral-sh/uv#7338)) - Avoid enforcing platform compatibility when validating lockfile ([#7305](astral-sh/uv#7305)) - Avoid installing transitive dev dependencies ([#7318](astral-sh/uv#7318)) - Avoid selecting prerelease Python installations without opt-in ([#7300](astral-sh/uv#7300)) - Fix PPC64 page size in binary builds. ([#7298](astral-sh/uv#7298)) - Include pre-release Python versions in `uv python list` ([#7290](astral-sh/uv#7290)) - Make version ID optional for source builds ([#7362](astral-sh/uv#7362)) - Support relative paths in `uv add --script` ([#7301](astral-sh/uv#7301)) ##### Documentation - Fix documentation typos for `uv build --build-constraint` flag ([#7330](astral-sh/uv#7330)) - Fix grammatical error in CLI docs ([#7353](astral-sh/uv#7353)) ##### Error messages - Add dedicated lock errors for wheel-only distributions ([#7307](astral-sh/uv#7307)) - Avoid treating `.whl` sources as source distributions ([#7303](astral-sh/uv#7303)) - Clarify Python requirement source for script incompatibilities ([#7339](astral-sh/uv#7339)) </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever MR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this MR and you won't be reminded about this update again. --- - [ ] If you want to rebase/retry this MR, check this box --- This MR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate).

zanieb force-pushed the zb/fix-run-pre branch from 806807e to 553325e Compare September 11, 2024 18:19

zanieb added the bug Something isn't working label Sep 11, 2024

This was referenced Sep 11, 2024

Add support for pre-releases in Python version requests #7270

Closed

Include pre-release Python versions in uv python list #7290

Merged

zanieb force-pushed the zb/fix-run-pre branch from 553325e to 243403b Compare September 11, 2024 18:43

zanieb commented Sep 11, 2024

View reviewed changes

This was referenced Sep 11, 2024

Add Python 3.13 system test for Linux #7302

Merged

Allow explicitly requesting an system interpreter version in check_system_python #7306

Merged

zanieb added a commit that referenced this pull request Sep 11, 2024

Allow explicitly requesting an system interpreter version in `check_s…

ebd73d8

…ystem_python` (#7306) Needed for #7300

Base automatically changed from zb/fix-list-pre to main September 11, 2024 19:46

zanieb force-pushed the zb/fix-run-pre branch from 243403b to 9a7a758 Compare September 11, 2024 19:46

Avoid selecting prerelease Python installations without opt-in

7dcaf00

Similar to our sementics for packages with pre-release versions, we will prefer non-prerelease versions unless there are only prerelease versions available.

zanieb force-pushed the zb/fix-run-pre branch from 9a7a758 to 7dcaf00 Compare September 11, 2024 19:51

zanieb commented Sep 11, 2024

View reviewed changes

charliermarsh approved these changes Sep 11, 2024

View reviewed changes

zanieb merged commit f22e5ef into main Sep 11, 2024
59 checks passed

zanieb deleted the zb/fix-run-pre branch September 11, 2024 20:49

BrewTestBot mentioned this pull request Sep 13, 2024

uv 0.4.10 Homebrew/homebrew-core#189400

Merged

hauntsaninja mentioned this pull request Sep 16, 2024

python -m uv no longer installs to the python specified (if prerelease) #7417

Closed

henryiii mentioned this pull request Sep 16, 2024

fix: support uv 0.4.10+ on Linux and update dependencies pypa/cibuildwheel#2008

Merged

henryiii mentioned this pull request Sep 16, 2024

ci: update to astral-sh/setup-uv pypa/cibuildwheel#2012

Closed

This was referenced Sep 17, 2024

Consider presence on the PATH an opt-in to Python pre-releases #7469

Closed

Allow creating venv with free-threaded python builds #7431

Merged

Rename PythonRequest::Any -> PythonRequest::Default #7514

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Avoid selecting prerelease Python installations without opt-in #7300

Avoid selecting prerelease Python installations without opt-in #7300

zanieb commented Sep 11, 2024 •

edited

Loading

zanieb commented Sep 11, 2024 •

edited

Loading

zanieb Sep 11, 2024 •

edited

Loading

zanieb Sep 11, 2024 •

edited

Loading

zanieb Sep 11, 2024

zanieb Sep 11, 2024 •

edited

Loading

zanieb Sep 11, 2024

henryiii commented Sep 16, 2024 •

edited

Loading

zanieb commented Sep 16, 2024

henryiii commented Sep 17, 2024 •

edited

Loading

		run: python3.13 scripts/check_system_python.py --uv ./uv
		run: python3.13 scripts/check_system_python.py --uv ./uv --python 3.13

Avoid selecting prerelease Python installations without opt-in #7300

Avoid selecting prerelease Python installations without opt-in #7300

Conversation

zanieb commented Sep 11, 2024 • edited Loading

zanieb commented Sep 11, 2024 • edited Loading

zanieb Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

zanieb Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

zanieb Sep 11, 2024

Choose a reason for hiding this comment

zanieb Sep 11, 2024 • edited Loading

Choose a reason for hiding this comment

zanieb Sep 11, 2024

Choose a reason for hiding this comment

henryiii commented Sep 16, 2024 • edited Loading

zanieb commented Sep 16, 2024

henryiii commented Sep 17, 2024 • edited Loading

zanieb commented Sep 11, 2024 •

edited

Loading

zanieb commented Sep 11, 2024 •

edited

Loading

zanieb Sep 11, 2024 •

edited

Loading

zanieb Sep 11, 2024 •

edited

Loading

zanieb Sep 11, 2024 •

edited

Loading

henryiii commented Sep 16, 2024 •

edited

Loading

henryiii commented Sep 17, 2024 •

edited

Loading